Repository navigation
Support test automation exclusions in JUnit5 - #12941
calvinbayer wants to merge 2 commits into
Conversation
2528d3f to
c943aca
Compare
CI Visibility Test Environment - sbt-scalatestJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
c943aca to
0b80b18
Compare
CI Visibility Test Environment - netflix-zuulJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - nebula-release-pluginJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - reactive-streams-jvmJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - heliboardJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - pass4sJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - sonar-kotlinJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - jolokiaJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - okhttpJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
CI Visibility Test Environment - spring_bootJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
🟢 Java Benchmark SLOs — All performance SLOs passed
PR vs. master results
Commit: Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion. |
CI Visibility Test Environment - sonar-javaJob Status: 🟢 success
Baseline: median of Tests run to verify that CI Visibility behavior has not regressed in the current PR. |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
More details
JUnit 5 skip selection receives declaration metadata consistently with the existing retry-policy and reporting paths. The completed static review identified no actionable regression against the supplied base.
🤖 Bits Code Review · Commit 0b80b18
0b80b18 to
9e1ec7c
Compare
9e1ec7c to
73c9c70
Compare
There was a problem hiding this comment.
The new exclusion resolver lacks coverage in internal-api’s own JaCoCo report, causing the required Java 8 coverage gate to fail despite tests in dependent modules.
🤖 Bits Code Review · Commit 9e1ec7c
Findings that could not be posted inline
internal-api/src/main/java/datadog/trace/api/civisibility/config/TestExclusions.java:12
Add owning-module coverage for the exclusion resolver
Java 8 CI runs baseTest with -PcheckCoverage, enforcing internal-api’s per-class minimums of 70% branch and 80% instruction coverage. This new helper has no module-local tests or coverage exclusion; tests added in other modules do not contribute to its report. The required coverage verification therefore fails. Add internal-api tests for the resolver and cover or explicitly exclude the newly executable TestEventsHandler.skipReason default method.
React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it
What each reaction means
- 👍 Useful: correct, and you're fixing it or will.
- 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
- 😕 Unclear: you can't tell what it means or why it matters.
- 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.
🤖 Bits Code Review · Open Bits AI session
|
Addressed the coverage finding in this AutoTest review with 8028762, which this PR inherits from its JUnit4 parent. The owning-module JaCoCo gate failed before the test additions: TestExclusions and the new default handler callbacks had zero coverage. Four durable behavior cases now cover independent annotations, tag resolution, empty/unknown metadata, and compatibility delegation through the legacy callbacks. On current master, the complete internal-api suite passes 5,256 tests and the coverage gate passes with 100% instruction and branch coverage for TestExclusions and 100% instruction coverage for the added TestEventsHandler defaults. No coverage thresholds were lowered. |
|
@autotest Review |
There was a problem hiding this comment.
More details
The JUnit 5 skip decision now receives declaration metadata and tags consistently with the existing retry-policy and reporting paths. The completed static review identified no actionable regression.
🤖 Bits Code Review · Commit 73c9c70
73c9c70 to
2d9d2d2
Compare
|
@autotest Review |
There was a problem hiding this comment.
More details
Method annotations and JUnit tags reach the exclusion-aware skip policy, consistent with the metadata already used for retries and reporting. Native skips and developer test selection retain their existing semantics.
🤖 Bits Code Review · Commit 2d9d2d2
2d9d2d2 to
db4f4d4
Compare
|
@autotest Review |
There was a problem hiding this comment.
More details
JUnit 5 supplies test-source metadata and tags to the existing exclusion-aware skip policy, enabling declaration-level protections during skip decisions. Runtime behavior remains unverified because tests were not executed.
🤖 Bits Code Review · Commit db4f4d4
db4f4d4 to
3cc0ab4
Compare
|
@autotest Review |
There was a problem hiding this comment.
More details
JUnit5 skip decisions now receive the declaration metadata needed to honor automation exclusions, consistent with execution policies and reporting. The completed specialist review identified no actionable regression in the supplied diff.
🤖 Bits Code Review · Commit 3cc0ab4
3cc0ab4 to
16de1ed
Compare
|
@autotest Review |
There was a problem hiding this comment.
More details
JUnit5 skip-policy selection receives declaration metadata while retaining the ITR unskippable guard. Static review identified no actionable defect; runtime behavior remains unverified because tests were inspected but not executed.
🤖 Bits Code Review · Commit 16de1ed
16de1ed to
308e8e1
Compare
|
@autotest Review |
There was a problem hiding this comment.
More details
Declaration metadata reaches the shared exclusion policy while preserving framework skips and ITR unskippable handling. Static inspection found no reportable defect in the JUnit 5 changes.
🤖 Bits Code Review · Commit 308e8e1
Shepherd verification — JUnit5 native tests and packaged Java smokeTested by: Shepherd Agent (autonomous QA for Datadog Test Optimization)
Jolokia is a TestNG packaged-agent smoke, not JUnit5 workload coverage. Native JUnit5 instrumentation tests verify the changed JUnit5 paths. This run uses Mockdog; no Datadog UI or production-backend validation is claimed. No backend API request/response changes were made. Issues found: No additional JUnit5 correctness issue was found. The inherited JUnit4 parameter-label regression was fixed in |
What Does This Do
Support test automation exclusions in JUnit5. Wire test declaration metadata into Datadog policy selection and reporting, preserving assigned Test Management properties.
Motivation
Allow individual tests to prevent Datadog quarantine/disabling and exclude Datadog ATR, EFD and Attempt to Fix. Developer selection and framework retries retain their existing semantics.
Additional Notes
Stacked on
codex/junit4-automation-exclusions; review this PR against its base for the framework-specific change. Shared public API and policy are introduced by the first JUnit4 PR.Validation: Module spotlessApply and instrumented JUnit5Test.test automation exclusions* passed, proving one execution, blocking failure, both annotation tags, preserved backend assignments, and no attempt-to-fix result.